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Abstract 

This paper studies the problem of optimal switching for one-dimensional diffusion, 
which may be regarded as sequential optimal stopping problem with changes of regimes. 
The resulting dynamic programming principle leads to a system of variational inequa- 
lities, and the state space is divided into continuation regions and switching regions. 
By means of viscosity solutions approach, we prove the smoot-fit C 1 property of the 
value functions. 
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1 Introduction 



In this paper, we consider the optimal switching problem for a one dimensional stochastic 
process X. The diffusion process X may take a finite number of regimes that are switched 
at time decisions. The evolution of the controlled system is governed by 

dX t = b(X t ,I t )dt + a(X t ,I t )dW t , 

with the indicator process of the regimes : 

It = K n l Tn <t<T n+ i ■ 

n 

Here W is a standard Bronian motion on a filtred probability space (O, J 7 , F = (J-t)t>o, P), b, 
a are given maps, (r n ) n is a sequence of increasing stopping times representing the switching 
regimes time decisions, and n n is J- Tn -measurable valued in a finite set, representing the 
new chosen value of the regime at time r n and until T n +i. 

Our problem consists in maximizing over the switching controls (r n , K n ) the gain func- 
tional 



E 



/ e-^f(x t ,i t )dt-^e- pTn g K 

Jo „ 
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where / is some running profit function depending on the current state and the regime, and 
gij is the cost for switching from regime i to j. We then denote by Vi(x) the value function 
for this control problem when starting initially from state x and regime i. 

Optimal switching problems for stochastic systems were studied by several authors, see 
PQ, [I] or [7j. These control problems lead via the dynamic programming principle to a 
system of second order variational inequalities for the value functions Vi. Since the Vi are 
not smooth C 2 in general, a first mathematical point is to give a rigorous meaning to these 
variational PDE, either in Sobolev spaces as in [I], or by means of viscosity notion as in 
[Zj. We also see that for each fixed regime i, the state space is divided into a switching 
region where it is optimal to change from regime i to some regime j, and the continuation 
region where it is optimal to stay in the current regime i. Optimal switching problem may 
be viewed as sequential optimal stopping problems with regimes shifts. It is well-known 
that optimal stopping problem leads to a free-boundary problem related to a variational 
inequality that divides the state space into the stopping region and the continuation region. 
Moreover, there is the so-called smooth-fit principle for optimal stopping problems that 
states the smoothness C 1 regularity of the value function through the boundary of the 
stopping region, once the reward function is smooth C 1 or is convex, see e.g. jS]. Smooth- 
fit principle for optimal stopping problems may be proved by different arguments and we 
mention recent ones in [2j or based on local time and extended Ito's formula. Our main 
concern is to study such smooth-fit principle in the context of optimal switching problem, 
which has not yet been considered in the literature to the best of our knowledge. 

Here, we use viscosity solutions arguments to prove the smooth-fit C 1 property of the 
value functions through the boundaries of the switching regions. The main difficulty with 
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regard to optimal stopping problems, comes from the fact that the switching region for the 
value function V{ depend also on the other value functions Vj for which one does not know a 
priori C 1 regularity (this is what we want to prove!) or convexity property. For this reason, 
it is an open question to see how extended Ito's formula and local time may be used to 
derive such smooth-fit property for optimal switching problems. Our proof arguments are 
relatively simple and does not require any specific knowledge on viscosity solutions theory. 

The plan of this paper is organized as follows. In Section 2, we formulate our optimal 
switching problem and make some assumptions. Section 3 is devoted to the dynamic 
programming PDE characterization of the value functions by viscosity solutions, through 
a system of variational inequalities. In Section 4, we prove the smooth-fit property of the 
value functions. 

2 Problem formulation and assumptions 

We start with the mathematical framework for our optimal switching problem. The stochas- 
tic system X is valued in the state space X C R assumed to be an interval with endpoints 
— oo < £ < r < oo. We let 1^ = {1, . . . , d} the finite set of regimes. The dynamics of the 
controlled stochastic system is modeled as follows. We are given maps b, a : X x 1^ — ► R 
satisfying a Lipschitz condition in x : 

(HI) \b{x,i) - b(y,i)\ + \a(x,i) - a(y,i)\ < C\x - y\, Vx, y G X, i G I d , 

for some positive constant C, and we require 

(H2) a(x,i) > 0, Vie int(X) = (£,r), i G I d . 

We set bi(.) = b(.,i), <7j(.) = i G 1^, and we assume that for any x G X, i G 1^, there 

exists a unique strong solution valued in X to the s.d.e. 

dX t = bi{X t )dt + <Ti(X t )dW t , X = x. (2.1) 

where W is a standard Brownian motion on a filtered probability space ($7, T, F = (J r t)t>0j P) 
satisfying the usual conditions. 

A switching control a consists of a double sequence ti, . . . , r n , . . . , «i, . . . , K n , . . ., n G 
N* = N \ {0}, where r n are stopping times, r n < T n +i and r n — > oo a.s., and n n is im- 
measurable valued in 1^. We denote by A the set of all such switching controls. Now, for 
any initial condition (x,i) G X x 1^, and any control a = (r n , n n ) n >i G A, there exists a 
unique strong solution valued in X x I d to the controlled stochastic system : 

X = x, I - = i, (2.2) 
dX t = b Kn (X t )dt + a Kn (X t )dW u I t = Kn, T n <t<r n+1 , n>0. (2.3) 

Here, we set To = and kq = i. We denote by (X x,t ,P) this solution (as usual, we omit 
the dependance in a for notational simplicity). We notice that X x,t is a continuous process 
and P is a cadlag process, possibly with a jump at time if t\ = and so Iq = k\. 

We are given a running profit function / : X x 1^ — ► R, and we assume a Lipschitz 
condition : 
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(H3) \f(x,i)-f(y,i)\<C\x-y\, Vx,yeX,ieI d , 

for some positive constant C. We also set /j(.) = f(-,i), i G Id- The cost for switching from 
regime i to j is constant equal to gij. We assume that : 

(H4) < g ik < + g jk , Mi^j^k^i G I d . 

This last condition means that the switching cost is positive and it is no more expensive to 
switch directly in one step from regime i to k than in two steps via an intermediate regime 
j- 

The expected total profit of running the system when initial state is (x, i) and using the 
switching control a = (r n ,K n ) n >i G A is 



J(x,i,a) = E 



/"OO 00 

/ e-*f{X x t \li)dt-Y,<r pTn 9K n -u>;, 

V° n=l 



where kq = i. Here p > is a positive discount factor, and we use the convention that 
e -pT n (u) _ q -^hgn r n (u;) = 00. We shall see below in Lemma 13.11 that the expectation 
defining J(x,i,a) is well-defined for p large enough (independent of x, i, a). The objective 
is to maximize this expected total profit over all strategies a. Accordingly, we define the 
function 

v(x,i) = sup J(x,i, a), x£LX, i £ 1^. (2-4) 

and we denote vi{.) := v(.,i) for i € 1^. The goal of this paper is to study the smoothness 
property of the value functions v j. Our main result is the following : 

Theorem 2.1 Assume that (HI), (H2), (H3) and (H4) hold. Then, for all i G I d , the 

value function Vi is continuously differentiable on int(X) = (£,r). 



3 Dynamic programming, viscosity solutions and system of 
variational inequalities 

We first show the Lipschitz continuity of the value functions V{ . 

Lemma 3.1 Under (HI) and (H3), there exists some positive constant C > such that 
for all p > C, we have : 

\vi(x) - Vi(y)\ < C\x-y\, Vx, y G X, i G I d . (3.1) 

Proof. In the sequel, for notational simplicity, the C denotes a generic constant in different 
places, depending on the constants appearing in the Lipschitz conditions in (HI) and (H3). 
For any a £ A, the solution to (|2.2|l - (|2.3|l is written as : 

Xf = x+ fb{Xfjl)ds+ f ' a{Xf,li)dW s 
Jo Jo 

CO 

A, = K nU n <t<r n+1 , (TO = 0, K = l). 

n=0 
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By standard estimate for s.d.e. applying Ito's formula to |X^' J | 2 and using GronwalFs 
lemma, we then obtain from the linear growth condition on b and a in (HI) the following 
inequality for any a € A : 



E 



X? 



< Ce ct (l + \x\' 2 ), t>0. 



Hence, by linear growth condition on / in (H3), this proves that for any a £ A : 



E 



dt 



CE 



e - pt (l + \X?' l \)dt 

POD 

< C / e' pt e ct {\ + \x\)dt 

Jo 

< C(l + \x\), 

for p larger than C. Recalling that the are nonnegative, this last inequality proves in 
particular that for all (x,i,a) £ X X Id X A, J(x,i,a) is well-defined, valued in [—00,00). 

Moreover, by standard estimate for s.d.e. applying Ito's formula to \X x,t — Xf' l \ 2 and 
using Gronwall's lemma, we then obtain from the Lipschitz condition (HI) the following 
inequality uniformly in a E A : 



E 



< e ct \x - y\ 2 , Vx,y € X, t> 0. 



From the Lipschitz condition (H3), we deduce 



sup E 
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f(x?>\ii)-f(xr,it 



dt 
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dt 



for p larger than C . This proves (|3.1|) . 



□ 



In the rest of this paper, we shall now assume that p is large enough so that from the 
previous Lemma, the expected gain functional J(x, i, a) is well-defined for all x, i, a, and 
also the value functions V{ are continuous. 

The dynamic programming principle is a well-known property in stochastic optimal 
control. In our optimal switching control problem, it is formulated as follows : 

Dynamic programming principle : For any (x,i) G X x I dl we have 



v(x, i) 



sup E 



f e- pt f(X^,r t )dt + e- pd v(X x /,P d )- £ e-^ 9K „_ liK „|3,2) 



where 9 is any stopping time, possibly depending on a £ A in (|3.2|) . This principle was 
formally stated in [X] and proved rigorously for the finite horizon case in [7j. The arguments 
for the infinite horizon case may be adapted in a straightforward way. 
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The dynamic programming principle combined with the notion of viscosity solutions 
are known to be a general and powerful tool for characterizing the value function of a 
stochastic control problem via a PDE representation, see We recall the definition of 
viscosity solutions for a P.D.E in the form 

F(x,v,D x v,Dl x v) = 0, x G O, (3.3) 

where O is an open subset in W 1 and F is a continuous function and noninceasing in its 
last argument (with respect to the order of symmetric matrices). 

Definition 3.1 Let v be a continuous function on O. We say that v is a viscosity solution 
to (j3.3|) on O it it is 

(i) a viscosity super 'solution to (|3.3|) on O : for any xq G O and any C 2 function if in a 
neighborhood of xq s.t. xq is a local minimum of v — <p and (v — p)(xo) = 0, we have : 

F(x ,ip(x ),D x (p(x ),Dl x ip(x )) > 0. 

and 

(ii) a viscosity subsolution to (|3.3|) on O : for any xq G O and any C 2 function tp in a 
neighborhood of xo s.t. xq is a local maximum of v — (p and (v — tp)(xo) = 0, we have : 

F(x ,cp(x ),D x ip(x ),Dl x (p(x )) < 0. 

We shall denote by Ci the second order operator on the interior (£, r) of X associated 
to the diffusion X solution to (12.11) : 



Cup = 7^o- 2 ip" + hip', i G I d . 

Theorem 3.1 Assume that (HI) and (H3) hold. Then, for eachi G 1^, the value function 
Vi is a continuous viscosity solution on (£, r) to the variational inequality : 



min <^ pvi - CiVi - /, , v { - max(vj - 9ij)\ = 0, x G (£, r). (3.4) 

This means that for all i G 1^, we have both supersolution and subsolution properties : 
(1) Viscosity supersolution property : for any x G (£, r) and p G C 2 (£,r) s.t. x is a local 
minimum of Vi — <p, Vi(x) = <p>{x), we have 



min { pp(x) - Ci(p(x) - fc(x) , Vi(x) - m&x(vj - gij){x) \ > 0, (3.5) 
I JV« J 

(2) Viscosity subsolution property : for any x G (£,r) and <p G C 2 (£,r) s.t. x is a local 
maximum of vi — <p, Vi(x) = p(x), we have 



min |p93(x) - Lnp{x) - fc(x) , Vi(x) - max(«j - fi^Xx) j < 0, 



(3.6) 
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Proof. The arguments of this proof are standard, based on the dynamic programming 
principle and Ito's formula. We defer the proof in the appendix. □ 

For any regime i 6 we introduce the switching region : 



S{ = < x G {£, r) : Vi(x) = max(u J - gij)(x) > . 
I J 

Si is a closed subset of (£, r) and corresponds to the region where it is optimal to change 
of regime. The complement set Ci of <Sj in (I, r) is the so-called continuation region : 



Ci = { x E (£, r) : v^x) > max(v i - 9ij)(x) } , 



where one remains in regime i. 



Remark 3.1 Let us consider the following optimal stopping problem : 



v(x) = sup E 

T stopping times 



[ T e-? r f(X?)dt + e-^h(X*; 
Jo 



(3.7) 



It is well-know that the dynamic programming principle for (|3.7[) leads to a variational 
inequality for v in the form : 

min {pv — Cv — f , v — h} = 0, 

where C is the infinitesimal generator of the diffusion X. Moreover, the state space domain 
of X is divided into the stopping region 

S = {x : v(x) = h(x)} , 

and its complement set, the continuation region : 

C = {x : v(x) > h(x)} . 

The smooth-fit principle for optimal stopping problems states that the value function v is 
smooth C 1 through the boundary of the stopping region, the so-called free boundary, once 
h is C 1 or convex. 

Our aim is to state similar results for optimal switching problems. The main difficulty 
comes from the fact that we have a system a variational inequalities, so that the switching 
region for Vi depend also on the other value functions Vj which are not convex or known to 
be C 1 a priori. 



4 The smooth-fit property 

We first show, like for optimal stopping problems, that the value functions are smooth C 2 
in their continuation regions. We provide here a quick proof based on viscosity solutions 
arguments. 
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Lemma 4.1 Assume that (HI), (H2) and (H3) hold. Then, for all i E 1^, the value 
function v-- L is smooth C 2 on Ci and satisfies in a classical sense : 

pvi(x) - CiVi(x) - fi(x) = 0, x £ Ci. (4.1) 

Proof. We first check that v i is a viscosity solution to (|4.1I) . Let x E Ci and ip aC 2 function 
on Ci s.t. x is a local maximum of Vi — <p, Vi(x) = <p(x). Then, by definition of Ci, we have 
Vi(x) > maxj^i(vj — gij)(x), and so from the subsolution viscosity property ()3.6I) of Vi, we 
have : 

pip(x) - Ci(p(x) - fi(x) < 0. 

The supersolution inequality for (J4.1|) is immediate from (J3.5D . 

Now, for arbitrary bounded interval (xi, 22) C Cj, consider the Dirichlet boundary linear 
problem : 

pw(x) - Ciw(x) - fi{x) = 0, on (21,22) (4.2) 

tu(2l) = Uj(2l), lt>(aj 2 ) = Vi(x 2 ). (4.3) 

Under the nondegeneracy condition (H2), classical results provide the existence and unique- 
ness of a smooth C 2 function w solution on (21, 22) to (|4.2|) - (|4.3[) . In particular, this smooth 
function it; is a viscosity solution of (|4.1|) on (21,22). From standard uniqueness results on 
viscosity solutions (here for a linear PDE in a bounded domain), we deduce that Vi = w 
on (21,22). From the arbitrariness of (21,22) C Cj, this proves that Vi is smooth C 2 on Cj, 
and so satisfies (|4.1j) in a classical sense. □ 

We now state an elementary partition property on the switching regions. 

Lemma 4.2 Assume that (HI), (H3) and (H4) hold. Then, for all i E Id, we have Si = 
Uj^iSij where 

Sy = {xeCj : Vi(x) = (vj - gij)(x)} . 

Proof. Denote Si = Uj&Sij. Since we always have Vi > m&Xj^i(vj — gij), the inclusion Si 
C Si is clear. 

Conversely, let 2 E Si. Then there exists j ^ i s.t. Vi{x) = 1^(2) — gij. We have two 
cases : 

-k if x lies in Cj, then x £ Sij and so 2 £ Si. 

•k if 2 does not lie in Cj , then 2 would lie in Sj , which means that one could find some 
k 7^ j s.t. Vj{x) = 1^(2) — gjk, and so ^(2) = Vk{x) — — g^. From condition (H4) and 
since we always have Vi > — g^, this would imply Vi(x) = ^(2) — g^. Since cost g^ is 
positive, we also get that k 7^ i. Again, we have two cases : if 2 lies in Cfc, then 2 lies in Sik 
and so in S{. Otherwise, we repeat the above argument and since the number of states is 
finite, we should necessarily find some I ^ i s.t. 1^(2) = V[(x) — gu and 2 € C\. This shows 
finally that 2 £ Si. □ 
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Remark 4.1 Sij represents the region where it is optimal to switch from regime i to 
regime j and stay here for a moment, i.e. without changing instantaneously from regime j 
to another regime. 

We can finally prove the smooth-fit property of the value functions Vi through the 
boundaries of the switching regions. 

Theorem 4.1 Assume that (HI), (H2), (H3) and (H4) hold. Then, for all i G I d , the 

value function v% is continuously differentiable on (£,r). Moreover, at x £ Sij, we have 
v'i(x) = v'jix). 

Proof. We already know from Lemma 14 . 1 1 that is smooth C 2 on the open set Cj for all 
G Id- We have to prove the C 1 property of Vi at any point of the closed set We denote 
for all j G Id, j 7^ i, hj = Vj — gij and we notice that hj is smooth C 1 (actually even C 2 ) 
on Cj. 

1. We first check that Vi admits a left and right derivative v[_{xq) and v' i+ (xo) at any 
point xo in «Sj = Uj^iSy. We distinguish the two following cases : 

• Case a) xo lies in the interior Int(iSj) of <Sj. Then, we have two subcases : 

★ xq G Int(5jj) for some j ^ i, i.e. there exists some 5 > s.t. [xq — 5, xo + 5] C Sij. 
By definition of Sij , we then have Vi = hj on [xq — 5, xq + 5] C Cj , and so Vi is differentiable 
at xq with v'^Xq) = hj(xo). 

-k There exists j ^ k ^ i in 1^ and 5 > s.t. [zo — ^^o] C 5ij and [xo,xo + 6] C 5,^. 
We then have Vi = hj on [xq — 5, xq] C Cj and i?j = hk on [xo, xo + 6] C Ck- Thus, Vi admits 
a left and right derivative at xo with v[ _(xo) = h'-{xo) and v ■ + (xo) = /i' fc (xo). 

• Case b) xq lies in the boundary dSi = Si\ Int(<Sj) of Si. We assume that xo lies in the 
left-boundary of Si, i.e. there exists 5 > s.t. [xo — 8, Xq) C Cj (the other case where xo 
lies in the right-boundary is dealt with similarly). Recalling that on Cj, Vi is solution to : 
pvi — Cvi — fi = 0, we deduce that on [xq — 5, xo), Vi is equal to Wi the unique smooth C 2 
solution to the o.d.e. : pwi — Cuii — fi = with the boundaries conditions : Wi(xQ — 5) = 
Vi(xo — 5), Wi(xo) = Vi(xo). Therefore, V{ admits a left derivative at xq with v[ _{x$) = 
w'^xq). In order to prove that Vi admits a right derivative, we distinguish the two subcases : 

★ There exists j 7^ i in I4 and 5' > s.t. [xo, xo + 8'] C Sij. Then, on [xo, xo + 5'], Vi is 
equal to hj. Hence Vi admits a right derivative at xo with v' i+ {xo) = h'j(xo). 

-k Otherwise, for all j 7^ i, we can find a sequence (x J n ) s.t. x J n > Xq, x J n £ Sij and x J n — > 
xq. By a diagonalization procedure, we construct then a sequence (x n ) s.t. x n > xq, x n ^ 
S^ for all j 7^ i.e. x n G Cj, and x n — > xo- Since Cj is open, there exists then 5" > s.t. 
[xo, xo + 5"] C Cj. We deduce that on [xo, xo + 5"}, Vi is equal to vbi the unique smooth C 2 
solution to the o.d.e. pwi — Cwi — /« = with the boundaries conditions Wi(xo) = Vi(xo), 
ii>i(xo + 5") = Vi(xo + 5"). In particular, Vi admits a right derivative at xo with v\ + (xo) = 
w'^xo). 

2. Consider now some point in Si eventually on its boundary. We recall again that from 
Lemma l4~2"l there exists some j ^ i s.t. xq G Sij : i>i(xo) = hj(xo), and hj is smooth C 1 on 
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xq in Cj . Since vj > hj , we deduce that 



Vi(x) - Vi(x ) ^ hj(x) - hj(x ) 



< J — -, V X < Xq 



X — Xq X — Xq 

Vi(x)-Vi(x ) ^ hj(x) - hj(x ) 



> — , M X > Xq, 

X — Xq X — Xq 

and so : 

v'i-(xo) < h'jixo) < vl + (x ). 

We argue by contradiction and suppose that V{ is not differentiable at xq. Then, in view 
of the above inequality, one can find some p G {v[ _(xq), v\ + (xq)). Consider, for e > 0, the 
smooth C 2 function : 

ip £ (x) = Vi(x ) + p(x - Xq) + -^(X - Xq) 2 . 

Then, we see that V{ dominates locally in a neighborhood of xq the function (p E , i.e xq is 
a local minimum of Vi — (p e . From the supersolution viscosity property of Vi to the PDE 
(J33J), this yields : 

PVe(xo) ~ Cicp e (x ) - fi(x ) > 0, 

which is written as : 

pviixo) - bi{xo)p - fi{xo) - —<Ti(x Q ) > 0. 

Sending e to zero provides the required contradiction under (H2). We have then proved 
that for xq G Sij, v'^xq) = h'Axo) = v'Axq). □ 



Appendix: Proof of Theorem 13.11 

(1) Viscosity supersolution property. 

Fix i G Id- Consider any x G (£, r) and ip G C 2 (£,r) s.t. x is a minimum of v i — ip in a 
neighborhood B £ (x) = (x — e, x + e) of x, e > 0, and Vi(x) = (p(x). By taking the immediate 
switching control t\ = 0, K\ = j ^ i, r n = oo, n > 2, and 6 = in the relation (|3.2j) . we 
obtain 



Vi(x) > Vj(x)-gij, Vjj^i. 



(A.l) 



On the other hand, by taking the no-switching control r n = oo, n > 1, i.e. J| = i, t > 0, 
X 1 ' 1 stays in regime i with diffusion coefficients bi and Uj, and 9 = r e A h, with /i > and 
t £ = mf{t > : Xf { $ B £ (x)}, we get from ^) : 



tp(x) = Vi(x) > E 
> E 



e-^fiiX^dt + e-^ViiX^) 
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By applying Ito's formula to e pt (p(X^ 1 ) between and 6 = r e A h and plugging into the 
last inequality, we obtain : 



-pt 



(pip -Cup- fi) (Xf' % ) 



0. 



Prom the dominated convergence theorem, this yields by sending h to zero : 

(pip -Cup - fi)(x) > 0. 
By combining with (jA.lJ) . we obtain the required supersolution inequality 
(2) Viscosity subsolution property. 

Fix i G Id, and consider any x G (£,r) and ip G C 2 (£, r) s.t. x is a maximum of v% — tp 
in a neighborhood B e (x) = (x — e, x + e) of x, e > 0, and Uj(x) = <£>(x). We argue by 
contradiction by assuming on the contrary that (J3.6|) does not hold so that by continuity 
of Vi, Vj, j 7^ i, <p and its derivatives, there exists some < 5 < e s.t. 

(p<p - Cup - fi)(x) > 5, VxeB s (x) = (x-5,x + 5) (A.2) 
Vi(x) — max(v j — gij)(x) > 5, Vx G Bg(x). (A-.3) 

For any a = (r ra ,K n ) n >i G A, consider the exit time t$ = inf{i > : X^' 1 ^ B$(x)}. By 
applying Ito's formula to e~ pt <p(X^' 1 ) between and 9 = t\ A Tg, we have by noting that 
before 6, X x ' 1 stays in regime i and in the ball Bs(x) C B £ (x) : 

Vi(x) = <p(x) = E / e-P t (pip-£ i <p)(X?' t )dt + e- p0 <p(XZ< 

Jo 

i 

e- p \pp - Ciip){X^)dt + e'^v^Xf 



> E 



(A.4) 



Now, since 9 = t$ A ti, we have 

e^v{Xf\r e ) - £ 9K n . u n n = [v(X^, Kl ) - g iKl ) l n < TS + e -^(X*/)l ra<n 

r„<9 

< e-r* (vi(X^) - 5) l Tl < Ts + e-™ Vi{X *f)l Ts<Tl 
= e'^Viixf) - 5e-^l Tl < T6 , 

where the inequality follows from ()A.3|) . By plugging into (|A.4j) and using (|A.2|) . we get : 



vt(x) > E 



Jo r n <9 

e-P'dt + e-P^l^ 



+ SE 



(A.5) 



We now claim that there exists some positive constant cq > s.t. : 



E 



e- pt dt + e- pT1 l 



Tl<T$ 



> cq, Va G A. 
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For this, we construct a smooth function w s.t. 



max.{pw(x) — Ciiv(x) — 1 , w(x) — 1} < 0, Vx G B$(x) 

w (x) = 0, Vx G 8B s (x) 

w(x) > 0. 

For instance, we can take the function w(x) = cq ( 1 — ) , v 



with 



{x : |x 



x = 



(A.6) 
51A.7) 
(A.8) 




U0 J 
from l)A,6|) . 1)A,7|) and (|A.8[) . By plugging this last inequality (uniform in a) into (|A.5|) . we 
then obtain : 



which is in contradiction with the dynamic programming principle (|3.2JI . 
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